


NPS55Sy74101 

NAVAL 



POSTGRADUATE SCHOOL 

Monterey, California 




SOME REMARKS ON THE FINITE-MEMORY 
K-HYPOTHESES PROBLEMS 

by 

Bruno 0. Shubert 
October 1974 

Approved for public release; distribution unlimited. 
Prepared for; 

Chief of Naval Research, Arlington, Virginia 22217 



■EDDOCS 

) 208.14/2:NPS-55Sy74101 




NAVAL POSTGRADUATE SCHOOL 
Monterey, California 



Rear Admiral Isham Linder 
Superintendent 



Jack R. Borsting 
Provost 



The work reported herein was supported by the Office of Naval Research 
through the Research Foundation Funds of the Naval Postgraduate School. 

Reproduction of all or part of this report is authorized. 

This report was prepared by; 



UNCLASSIFIED 



security classification of this page (Wht>n DmtB Entered) 



REPORT DOCUMENTATION PAGE 



READ INSTRUCTIONS 
BEFORE COMPLETING FORM 



1. REPORT NUMBER 

NPS55Sy74101 



2. GOVT ACCESSION NO 



3. RECIPIENT'S CATALOG NUMBER 



4 T\JLE (and Subtttie) 



Some Remarks on the Finite-Memory 
K-Hypotheses Problems 



5. TYPE OF REPORT & PERIOD COVERED 

Technical Report 



6. PERFORMING ORG. REPORT number 



7. AUTHORrs; 



Bruno 0. Shubert 



6. CONTRACT OR GRANT NUMBER^a; 



9. PERFORMING ORGANIZATION NAME AND ADDRESS 

Naval Postgraduate School 
Monterey, California 93940 



10. PROGRAM element. PROJECT, TASK 
AREA & WORK UNIT NUMBERS 



11. CONTROLLING OFFICE NAME AND ADDRESS 

Office of Naval Research, Arlington, VA 22217 



12. report date 

October 1974 



13. NUMBER OF PAGES 

28 



14. MONITORING AGENCY NAME ft AODRESS(lf different from Controlling Office) 



15. SECURITY CLASS, (of thia report) 

UNCLASSIFIED 



15a. DECLASSI FI cation/ DOWN GRADING 
SCHEDULE 



16 distribution statement (ot this Report) 

Approved for public release; distribution unlimited. 



“ 1 



17. DISTRIBUTION STATEMENT (of the abatract entered In Block 20, if different from Report) 



18. SUPPLEMENTARY NOTES 



9. KEY WORDS (Continue on reverae aide if neceaaary and identify by block number) 

Finite memory, Markov chains 



20. ABSTRACT (Continue on reverae aide If neceaaary and identify by block number) 

Finite-memory statistical problems typically deal with the situation where 
the class of statistics is restricted to those taking on a fixed finite number 
of values. Although a potentially infinite number of samples may be available 
the statistician is allowed to base his inference only on the current value 
of such a statistic - the current state of his finite memory. This is the 
case for instance when the inference is to be performed by a small size compute! 

During the past several years a number of results have been obtained con- 
cernln g a t wo-hyp otheses finite-me mory problem. In this report we consider 



DD 



FORM 
AN 73 



1473 EDITION OF 1 NOV 65 IS OBSOLETE 

S/N 0102-014- 6601 ' 



security classification of this page (Whan Data Bntmrad) 



^LCURITY CLASSIFICATION OF THIS PAGErWTian Data Bntarad) 



20. some aspects of the case where the number of hypotheses is greater than 
two. In particular we derive a bound on the error probability for the 
3-hypothesis case, present a counterexample to a recently proposed 
conjecture and briefly discuss a finite-memory version of the minimax 
theorem. We also include two appendices containing some results on 
finite Markov chains. 



security classification of THIS P^GE(Whan Data Bntarad) 



1 . INTRODUCTION 



Let X^, be a sequence of independent, identically distributed 

random variables taking on values in some measurable space X . Let 

K 

be a finite collection of probability measures on X , and let H ,k = 1,...,K , 

iC 

denote the hypothesis that the common distribution of the X ’ s is P, . 

n k 

We wish to associate with the observed sequence X^, X 2 ,... a sequence of 

decisions d^,d , , d } about the true hypothesis H. However, 

j- z n X K 

the decision d^ at time n is allowed to depend on X^,...,X^ only through 
a finite-valued statistic T^€ {l,...,m} , which represents the current state 
of the memory. This statistic is updated after each observation, i.e., 

T = f(T ,X ) , n=l,2,... , 

n-rl n n 

where f : {l,...,m} x X ^ {l,...,m} is a time-invariant updating rule. The 
decision d^ is then given by 

d = d(T ) , n=l,2 , ... , 

n n 

where d : {l,...,m} } is a time-invariant decision rule. 

i K 

Let for a given f and d 

= lim I I P(d^ # Hj^) (1.1) 

n=l 

be the asymptotic expected frequency of incorrect decisions if the true 
hypothesis is . Our goal is to find (f,d) which minimizes 



.,K 



,(k) 



e 



(f,d) 



P (f,d) = max 
^ k=l,.. 



( 1 . 2 ) 



(Thus we have chosen the minimax criterion. An alternate approach would be 
to minimize 

P^(u;(f,d)) = I 7T , 

® k=l 

where it = (tt , ) is a prior distribution on {H , . . . ,H } . In this 

i K IK 

report we consider the former.) 

The pair (f,d) together with the domains and ranges of the two functions 
is formally equivalent to the definition of a finite automaton (see e.g. [1]). 

The automaton has S = {l,..,,m} as its state space, X as its input space, 

} as its output space, and f and d as its state-transition 

i K 

function and state-output function respectively. If the sequence . 

of i.i.d. random variables is applied to the input of such an automaton the 
resulting sequence of states T^,!^,... is then a time-homogeneous Markov 
chain with transition probabilities 

p..=P,({x€X:f(i,x)=j}), i,j£S. (1.3) 

IJ K. 

Hence the limit in (1.1) always exists. If the state-transition function f 
is such that the resulting chain is regular then in fact 

P^’^hf,d) = , 

where y is the stationary distribution on S . Throughout this paper we 

K 

assume that this is the case, i.e,, we consider only^ transition functions which 
yield regular Markov chains under each hypothesis. 

Following Heilman and Cover [3] we would like to include the possibility 
that the transition function f can be randomized. One way of defining such 
a randomization would be to introduce another input sequence • • • of 



2 



i.i.d. random variables, independent of the sequence and uniformly 

distributed on the interval [0,1]. The transition probabilities (1.3) would 
then be 

p.. = E, {p..(X)} , where (1.4) 

Pi^(x) = A({y€ [0,1] : f(x,y,i) = j}) , (1.5) 

A being Lebesgue measure on [0,1]. 

However, we find it more convenient to express the randomized state transition 
function f as a pair (A, A) as follows: 



A = 




i=l,...,m ; j=l,...,m ; iA j ) , 



where A. . are measurable subsets of X . 
ij 



: i=l,...,m ; j=l,...,m ; iA j } , 

where 6. . > 0 and J 6. . < 1 for all i,j . 
xj xj 

The transition probabilities (1.5) if X = x is observed are now defined by 



P..(x) =6.. whenever A. . 9 x for iAj , 
ij iJ iJ 



P..(x) =1-1 ’ 

XI jAi 



and (1.3) becomes 



p. . = P, (A. ,)6 . . if iAj , 

^xj k ij ij 



Pii 



= 1 - I 



( 1 . 6 ) 



^xj 



We will refer to the triplet (A, A , d) as randomized finite automaton (RFA) 
and to the set A as randomization. 



3 



Notice the class of all randomization is closed with respect to multi- 



plication of corresponding elements, that is if 

are randomization then A A* = {6.. 6 *..} is again a randomization. Notice 

ij 

also that the sets A., need not be disioint. 

We now present a simple lemma to be used in the remaining sections. 
Lemma 1 : Let (A,A,d) be a RFA, let , k = 1, . . . ,K , be stationary 

distributions of the resulting Markov chain of states. Let R = t>e a 

K X K matrix with positive entries 






let p(A,A,d) be the maximal eigenvalue of R 
Then 



^ ^ - p(A,A, d) ■ 



and there exists a randomization A' such that 



P (A,A A ' , d) = 1 - --77- ^ — jv . 
e" ’ p(A,A ,d) 



Proof : 



(1-P ) ^ = (1-max P ^ 

^ k ^ 

= max (1-P ^ = max (y (d ^(H ))) ^ 

, e , k k 

k k 

K 



= ">ax r > p , 



1 « -1 
k £=1 

since by Perron-Frobenius theorem the maximal eigenvalue of a positive matrix 
can never exceed the largest of the row-sums . 
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To prove the second statement let v = (v , ...,v ) be an eigenvector 

1 K 

corresponding to p(A,A ,d) and normalized such that 



v^ > 0 , k = 1 , . . . ,K , v^ + v^ + • • • + v^ = a , 

where 0 < a < 1 is an arbitrary constant. (This is always possible since the 
matrix R is positive.) Define the randomization a’ = ^^ij^ 

6.. = ~ , i 7 >^i 5 where u. = v, whenever i€d ) . 

11 u. 1 k k 

1 

(k) (k) 

Let p^. and p^ . be transition probabilities and and y’ the sta- 

tionary distribution of (A, A,d) and (A, A A ’ ,d) respectively. Then by 
(1.6) for any k and 1^2 

= J:. 

^ . u . ^ij * 

and hence for any partition of the state space S into two subsets and 

we must have 



I I = I I 

i€S^ j€S2 ^ iCS2 j€S^ 



(k). 



i€S^ 265^ i€S^ j€S^ 



Thus 



CkUiUk(i) all i > k with Ck > 0 independent of i so that 






K . But then for all k 
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for all k,i = 1, . . . , 



Q.E.D. 



K 



K 






^=1 



i,=i 



since v is an eigenvector of p(A,A,d) . 



2. UPPER BOUND ON THE ERROR PROBABILITY FOR THE 3-HYPOTHESES 3-STATE PROBLEM 
Let K = 3 , m = 3 , and let 

P* = inf P (A, A ,d) , 
e e 

the infimum being taken over the class of all 3~state RFA. Let for i,j = 1,2,3 



sup P . (A) /P. (A) 
ACX ^ ^ 

^ij inf P.(A)/P.(A) 
ACX ^ ^ 



( 2 . 1 ) 



let 



g = (1/3) (y ^2 ^23 ^13^ ’ 



-1 -1 



5 =(l/2)max{Y^^ + y^^ , y'^ + , y"^ + Y' 3} 



In this section we show that 



P* < 1 - (1 + 2g cosh(l/3 argcosh G g 



-1 



( 2 . 2 ) 



We also establish a simpler but looser bound, namely 



P* < 1 - (1 + 2G^/^)'^ 
e 



(2.3) 



Notice that (2 J) implies that if Y 32 ~ Y23 “ Y33 = + “ then P^ = 0 , a result 
obtained by Sagalowicz in [4] and extended later by Yakowitz in [5]. 
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Proof of (2.2) and (2.3): 



Let the letters i,j,k stand for a permutation of 1,2,3, let e > 0 . 
Let A (e) be measurable subsets of X such that 



let 



Pi(A_(e)) P^(A) 

Pj(A.j(e)) - ’ 






P.(A..(e)) P^(A.^(e)) 

P,(A..(e)) P.(A..(e)) 

1 ij J Ji 



so that Y..(e)'^Y.. as e->0. 
ij ij 

A = {A..(e)} , A = {6..} such that 
ij ij 

d(i) = H. , d(j) = H. , d(k) = H, . 

1 J K 



Consider a 3-state RFA (A, A ,d) , where 

6 =6 = 0 , otherwise arbitrary, and 

ik ki 



cr©: 



P(A. .)6. . 
IJ IJ 



P(A. ,)6.. 



P(A.. )6., 
jk jk 



©' 

u 



to 






kj 



The stationary distribution of the resulting Markov chain of states are 
(see Appendix A) 

M(i) = CP(A..)6^^P(A^.)6^^ . 

P(J) - cP(A^.)6^.P(A^j)6^j . 

U(k) - cP(Aj^) 6 ,^P(Aj.) 6 .j . 

where c is a normalizing constant. (The epsilon has been dropped temporarily 
to ease the notation.) The matrix R of Lemma 1 is given by 



7 



1 



R = 



P . (A . . ) 6 . . 

_J .11 JJ- 

P. (A. .)6. . 
1 iJ ij 






P. (A. .)6. . 
1 13 13 






P.(A., )P.(A. .)6., 6. . 

3. jk 1 11 jk ij 



’ P.(A.,)6.. ’ P.(A. .)P.(A, .)6. .6, . 

1 Ji Ji 1 Ji 1 Icj ji kj 






and its characteristic equation has the form 



(1-X)'" - (1“X)C + D = 0 , 

e e 



where 



and 



C 

e 




(e) + + 



^ 1 ^^,1 k^ ^ 1 ^‘^1.1 ^ ^k 1 ^ ^k J ^ 



D 

e 



P.(A. .)P.(A., )P, (A. .)P, (A, .) 
= 1 11 1 lk k 11 k Xl 

P.(A,.)P.(A. .)P, (A., )P, (A. .) 
1 ji j kj k jk k ij 

, P.(A..)P, (A, .)P.(A., )P.(A. .) 
+ 1 11 k ki 1 ik 1 n 



P.(A. .)P, (A., )P.(A. ,)P.(A, .) 
j ij k jk 1 ji X kj 



Now D can be written as 
e 



D = 



Y‘^e)Y"^e) 



, - 1 , , -1, , 



(2.4) 
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and hence 



D 

e 






Next writing C as 



C 

e 



- ^I3<=> 



+ yTJce) 



+ Y^^(e)Fj(£) 



where 



P. (A. .)P.(A., )P.(A, .)P, (A, .)P, (A. .)P, (A., ) 

= 1 ij 1 ,ik 1 Ici k^ kj k^ ji k^ ik'^ 

j ■ Pi(A.i)P.(A^.)P.(A,JP,_(A,JP,,(A,JP,.(A,J ‘ 



ik k jk k ij k ik 



it is seen that by setting i,j,k equal to the three cyclic permutations of 
1,2,3 we must have 

¥^(e)¥^(e)¥^(e) = 1 . 

Hence i,j,k can be chosen such that 






Now it is easily verified that the maximal root of the equation (2.4), which is 
real and not smaller than 1, is an increasing and continuous function of both the 
coefficients and . Thus by Lemma 1 there is for every e > 0 a 3-state 

RFA for which the error probability 

P < 1 - + e , (2,5) 

e 

where r is the maximal root of the equation 

(l-X)^ - (I-X)Cq + Dq = 0 , (2.6) 



C 

0 



-1 

''12 



-1 



-1 



1'23 + 1'13 
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with 



and 



n _ r -1 ^ -1 - 1^-1 -1 ^ 

Dq j_2 + Y23 > Y32 ”^13 ’ ^23 "^13 ' 



Clearly (1/3)0^ < (1/2)0^ and since (1/2 )Dq 1 we must have ((1/3)0^) < ((1/2 )Dq) 



Hence the maximal root of the cubic equation (2.6) is given by 



1/2 



r = 1 4* 2 ((1/3 )Cq) cosh(l/3)({) , 



,-3/2 



where cosh (p = 1/2 Dq((1/3)Cq) and the bound (2.2) follows from (2.5). The 



3 2 

simplified bound (2.3) can be obtained by increasing Cq until ((1/3) Cq) =((1/2)Dq) 



the maximal root of (2.6) thus becoming 



r = 1 + 2((1/2 )Dq) 



1/3 



3. COUNTEREXAMPLE TO TREE-CONJECTURE . 



Consider a K-hypotheses problem and assume for simplicity that the support 



of each of the distributions is same. With each RFA (A,A,d ) we can now 



associate a graph T with vertices corresponding to states of the RFA and with 



an arc joining vertices i and j if and only if ^ 0 . (This property 



does not depend on the hypothesis because of our assumption.) 



Let e > 0 , be the class of all e-optimal RFA, i.e., all m-state RFA 



(A, A ,d) such that 



P (A, A ,d) ^ inf P (A, A , d) + e . 
e e 

It has been conjectured by Cover [2] that for every e > 0 the class always 

contains a RFA whose graph P is a tree. This is indeed true for K = 2 ([3]) 

and a plausible heuristic argument can be given for such a structure even for 
K > 2 . Unfortunately, as we are going to show in this section, the conjecture 
is false already for K = 3 . We do this by exhibiting a nontrivial 3-hypotheses 
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problem and a 3-state RFA with a triangular graph T , which is strictly 
better than any 3“State RFA whose graph is a tree. 

Let X = {1,2, 3, 4, 5, 6} , let p,q,r,s be positive numbers such that 

2p + 2q + r + s = l , 

and 

1 < i « ^ . 

s q 

Define the three distributions Pj^,P 2 »P 2 as follows: 



P^(K) 


X 


1 


2 


3 


4 


5 


6 




1 


p 


q 


p 


q 


s 


r 


k 


2 


q 


p 


r 


s 


p 


q 




3 


S 


r 


q 


p 


q 


p 



Consider now a RFA (A, A ,d) with the state space S = {1,2,3} , d(i) = > i^S 

and the graph T the tree (l) — (g) — ( 5 ) . The matrix R for this RFA is the same 
as that on page 8 with (i,j,k) = (1,2,3) . Since by Lemma 1 the error probability 
is determined by the maximal eigenvalue p of R and p is always at least as 
large as the smallest row-sum, in order to minimize p we are forced to choose 
A as follows: 

A^2 = {2} , A^^ = {1} , A^3 = {6} , A32 = { 5 } . 
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The matrix R then becomes 



R = 






p 5 






21 



P" ‘^21 S 2 



q^21 



q ^23 



P 6 



12 



P ^ 



32 



6., 6, 



qs 21 32 



q *^32 



P^ *^23 ^12 



P 5 



23 



Writing its characteristic equation again as 



(1-X) - (l-A)C + D = 0 



we have 



C = 3(^)^ . D = (§ + 



r s 



(3.1) 



There are two other 3-state RFA whose graph P is a tree, one with the graph 

® ® (D with the graph (^) . By the same reasoning 

as before we are forced to choose 



Af 2 = {2} . A^^ = {1} , A ^3 = {4} , A 3 ^ = {3} 



for the foimer, and 



Ai 3 = {4} , A 33 = {3} , A ^3 = { 6 } , A 32 = {5} 



for the latter. The matrices R are 



1 ^ q 

’ P ’ P 



q 1 qs 

p ’ ’ 



pr 



q ££ 

P ’ ps 



12 



and 



51 

pr 



ps 



3 . £ 

P ’ P 



£ 

P 



£ 

P 



respectively, where we omitted the 6's for the sake of simplicity. Hence 
the coefficient of their characteristic equations are again given by (3.1). 
Now consider a 3-state RFA (A,^ ,d) with 



A^2 = {2} , = {1} , A^2 = {4} , A^^ = {3} , A^3 = {6} , A^^ = {5} , 

= 1/2 for all i?^j and d(i) = , i = 1,2,3 . The graph F of this RFA 

is a triangle. The matrix R is 





1 


£ P+2s 


£ 


p+2r 






p p+r+s 


’ P 


p+r+s 


£ 


p+2r 


1 


£ 


p+2s 


P 


p+r+s 




P 


p+r+s 


£ 


p+2s 


q p+2r 




1 


P 


P+r+s 


’ p p+r+s 


9 


X 



and the coefficients of its characteristic equation 



C . 3(1) 



2 p+2r p+2s 



p+r+s p+r+s 



= 3 (^)^ 



(1 - ( 



r-s 

p+r+s 






and 



p p+r+s 



+ ( 



P+2s . 3, 

p+r+s ^ 



13 



and 



Comparing these expressions with (3.1) we see that C 



n < C 

triangle tree 

. (Choose, for 

ree 

= 10^) . Since the 



with r and s suitably chosen also D 

^ tr 

-1 -3 

instance r = 10 p , s = 10 p . Then D 



< D 

triangle tree 

D /D . , 

tree triangle 



maximal eigenvalue increases with both C and D we conclude from Lemma 1 



that the best **tree" RFA has an error probability strictly larger than this 
’'triangular'* RFA. 

4. MINIMAX THEOREM FOR FINITE-MEMORY PROBLEMS . 

Let 7T = ( > • • • > be a probability distribution on the set of hypotheses, 
1 K 

let (A, A ,d) be RFA, and let this time the error probability be 



Looking now at the problem as a two-person zero-sum game, where the 1st player 
(Nature) chooses and the 2nd player (Statistician) chooses (A, A ,d) it 
is natural to ask whether 



,d)) = \ TT p^^^(A,a ,d) . 

e i ^ K. e 



k=l 



(A, A ,d) 7T_ 



inf 



sup P (tt;(A,A ,d)) = sup 

_ ^ 7T 



up inf 
^ (A, A ,d) 



P (tt;(A,A ,d)) 
e ” 



(4.1) 



Now if K - 2 then it is known [3] that 




(4.2) 



where is given by (2.1) . Hence 
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On the other hand by Lemma 1, 



inf sup = 1 - (sup p(A,A,d)) 



and for K = 2 it is easily seen that 



p (A, A , d) = 1 4- 






1/2 



However it has also been shown in [3] that 

It T " 

U^(d ^(H^))u2(d~-^(H^)) Yi2 

and hence (4.1) is indeed true for K = 2 . 

Conjecture : (4.1) is also true for K > 2 . 

Comment : Since an analog of (4.2) for K > 2 is not available at present the 

above reasoning cannot be applied to prove the conjecture. However, since the 
number of hypotheses is finite (4.1) would follow if one could show that the 
set of all vectors 



,d),...,P^‘^^A,A ,d)) , 

e e 

where (A, A ,d) runs through all m-state RFA, is convex. This is indeed so for 
K = 2 , unfortunately we have not been able to prove this even for K = 3 . 
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APPENDIX A 



A Formula For A Stationary Distribution 
Of A Finite Markov Chain. 

Let P = be an m x m. stochastic matrix, let g = (S,E) be an 

oriented graph with the set of vertices S = {l,...,m} and the set of arcs 
EC S X S defined by 

(i,j)€ and p_ > 0 . 

Let i£ S be a vertex of g. Then a vertex S such that (i, j) € E 
is called a successor of i . A sequence of vertices (i^ , i^ * • • • , i^) such 
that each ^ successor of i^ , k = l,...,n-l , is called a path . 

If i^ is also a successor of i the path (i^,...,i ) is called a cycle. 

Consider now a subgraph f = (S,F) , where FC E with the following 
properties . 

1) each vertex i€S has at most one successor. 

2) f has no cycles. 

3) f is maximal, i.e., no further arcs can be added without violating 
1) or 2). 

We will call such a subgraph a confluence . Notice that each confluence 
has exactly one vertex with no successor. We will refer to this vertex as a 
sink . 

With each confluence f = (S,F) we associate a positive number 

p(f) = n p 

We now have the following theorem; 
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Theorem : Let P be a transition probability matrix of a homogeneous Markov 



chain, g be the graph defined above, let (j)^ be the set of all confluences 
with sink i € S . 

If P has an invariant distribution )P then 

1 m 1 m 



y = C I p(f) , i€ S , (A.l) 

where C > 0 is a normalizing constant determined from p- + • . . + y =1. 

1 m 

Remark : Notice that the formula (A.l) gives y^ as a sum of products of the 

off-diagonal entries of P , each product contains exactly m - 1 different 
entries and no two products contain the same set of this sense, 

the representation of y^ is unique. Notice also, that if all off-diagonal 
entries of P are positive then y^ is a sum of exactly m^ ^ products. 

Thus, although the formula is certainly of theoretical interest, its application 
for computing the stationary distribution is likely to be limited to cases, 
where a majority of P^j*^ zero. 

Example : Let 



.5 


.2 


.3 


0 


.3 


.1 


.2 


.4 


.1 


.7 


.2 


0 


0 


.5 


0 


.5 



The graph g is 



and the confluences together with the number p(f) are as follows: 
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Sink 

i 



I p(f) 



Confluences f€ 



1 



2 



3 



3 



3 



1-2 4 

.015 



3 



1 2 
.105 



3 



4 



1 2 4 

.010 

3 



1 2-4 

.070 

3 



1 2 
.045 



2 4 

.030 



3 



12 4 

.010 



3 



12 4 

.105 

3 



12 4 

.020 



4 



3 



3 




3 

12 4 

.084 



Total: 



Hence the stationary distribution is £ 



130 185 95 148 

558 ’ 558 ’ 558 ’ 558 



.130 

.185 

.095 

.148 

.558 
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Proof of the Theorem: 



We will show that (A.l) satisfy the equations 



m 



J i=l ^ 



or equivalently 



m 



m 



y. I 

J i=i 






j Xj...,ni 



Let h = (S,H) be an arbitrary subgraph of g , let 



(A. 2) 



p(h) = n p 

(i,j)€ H J 

and let h + (i,j) denote a subgraph obtained from h by adding or removing 
the arc (i,j). Next let 



Aj = {f + (j,i) : f€ S - {j}} , 

= {f + (i,j) : - {j}} . 

If is given by (A.l) then for any J€ S 

m 



p. I Pi-i = I 

^ i=l h€A. 



and 



m 



J PiPii = I p^i^) 



i=l 



h€B. 
J 



Thus (A.l) will follow if we show that = B^ . To this end let 



h € Aj , h = f + ( j , i) , 
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let k be a vertex contained in the path (i,.,.,j) whose successor is j. 

If the arc (k,j) is removed then h becomes a conluence with sink k 
since (k,j) was an arc of confluence f and thus could not have any other 
successor than j . Hence h can be written as f’ + (k,j) , f’£ (J)j^ and 
therefore ^ ^ ^ j • Conversly, if ^ ^ ^ j > h = f* + (k,j) , then 

f'£ and by removing the arc (j,i) with i being a successor of j con- 
tained in the path (j,,,,,k) we conclude that h - (j,i)€4)j- Hence ^ ^ j 

and the proof is complete. 
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APPENDIX B 



A Generalization of a Lemma of Yakowitz ([5]). 



Lemma : Consider K finite regular Markov chains with state spaces , 

transition probabilities [Pj^(i->-j)] , and stationary distributions , 

k = 1,...,K . Link these chains together by allowing transition between 
and , k = 1,...,K - 1 , via a pair of states S 

with probabilities 



^^\,k+l \+l,k^ \,k+l ’ 

^^\+l,k \,k+l^ " \+l,k ’ 

and changing the original transition probabilities ®k,k+l^ 

PT_Li(e, ,1 , ) accordingly. 

k +1 k+l,k k+l,k 

If the new chain with state space S = U • • • U regular then its 

stationary distribution is given by 



k -1 



€ S^=^.,(s) - cp^(s),n^ 



K 

n TT. ,^y.(e. 9 k— I 5 ... 5 K, 



j=k+l 



j>j-l j jJ -1 



where C > 0 is a normalizing constant. 

Proof (by induction on K) 

(i) Let K = 2 . We have for the original two chains 



s€ S (s) = I P (r-vs)y (s) , 

^ ^ r€S^ 
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s€ S2=>M2 (s) = I P^Cr-^s) M2 (s) , 
r€ $2 



and for the new chain ~ ^2 ’ 



s£ S^,s ej^2==^ ^ ^ (r^) y^^2 > 



r€S, 



s€ S,,s 62^-=^y^2(®^ I P,(r-^)y^ ,(s) , 



r£S, 



12 



^12^®12^ \ P^(r->s)y^2(s) + ’^12^ ^12^®12^ 

r€Sj^-{e^2J 

■"*2l''l2<®21> ' L ‘’l<'^^>>‘l2< = > 

r«Sj 

since ’^2l'^12 ^®21^ ~ ^12^12^^12^ equating flows. Similarly 



^12^^21^= P2(r--s)y^2<®> * 

r£ ^2 



Hence if s £ S^ then y^^Cs) and satisfy the same system of linear 

equations and consequently 



s £ a^y^(s) 



s£S2=^y^2^^^ a2y2(s) . 



In particular 



and since 



^^12^12 



'12 



<^2> 



(ei2> - ^x^l^®12^ ’ ^12^®21^ ^2^2^®21^ 

= ■'^21^12^^21^ 



^2 ’^12 ^1^®12^ 
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Since = 1 this implies a^ = ’ ®2 ^ ’i2'^1^^12^ ’ 

(ii) Let the lemma be true for S = S, U...U S and form new chain SUS . 

i. K"“i K 

Denoting y' the stat. distribution of S and y that of SUS we have 
— K 

by part (i) 



s€S=>y(s) = C K-1*^K^\ 
S«s^=s.u(s) . C 



(B.l) 



By induction hypothesis 



K-2 






and if s € S, C S 
k 



k-1 



K-1 






Substitution into (B.l) gives the desired formula (with the proportionality 
constant CC). 
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